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Abstract 



For the Hermitian inexact Rayleigh quotient iteration (RQI) , the author has estab- 
lished new general convergence results, independent of iterative solvers for inner linear 
systems. The theory shows that the method converges at least quadratically under a 
new condition, called the uniform positiveness condition, that may allow inner tolerance 
£fc > 1 at outer iteration k and is fundamentally different from the condition < £ < 1 
with £ a constant not near one commonly used in literature. In this paper we first con- 
sider the convergence of the inexact RQI with the unpreconditioned Lanczos method 
for the linear systems. Some attractive properties are derived for the residuals of the 
linear systems obtained by the Lanczos method. Based on them and the new general 
convergence results, we make a more refined analysis and establish convergence results. 
It is proved that the inexact RQI with Lanczos converges quadratically provided that 
£,k < £, with a constant £ > 1, that is, the linear systems are allowed to be solved with 
no accuracy in the sense of solving the linear systems. The results arc fundamentally 
different from the existing quadratic convergence results and have an impact on effec- 
tive implementations of the method. We extend the new theory to the inexact RQI 
with a tuned preconditioned Lanczos for the linear systems. Based on the new theory, 
we can design practical criteria to best control to achieve quadratic convergence 
and implement the method more effectively than ever before. Numerical experiments 
confirm our theory. 
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1 Introduction 

We consider the problem of computing an eigenvalue A and the associated eigenvector x of 
a large and possibly sparse Hermitian matrix A £ C nxn , i.e., 



Throughout the paper, we are interested in the eigenvalue Ai closest to a target a and its 
corresponding eigenvector x\ in the sense that 
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Ax = Ax. 




|Ai — a\ < | A2 — c| < • • • < |A n — a 




Suppose that a is between Ai and A2. Then we have 

|Ai-a| < -|Ai-A 2 |. (3) 

We denote by x\, x<2, . . . ,x n are the unit length eigenvectors associated with Ai, A2, . . . , A n . 
For brevity we denote (X\,xi) by (A, x). There are a number of methods for solving this 
kind of problem, such as the inverse iteration [T3], the Rayleigh quotient iteration (RQI) 
|13j . the Lanczos method and its shift-invert variant [13j . the Davidson method and the 
Jacobi-Davidson method [21^ I22j . However, except the standard Lanczos method, these 
methods and shift-invert Lanczos require the exact solution of a possibly ill-conditioned 
linear system at each iteration. This is generally very difficult and even impractical by a 
direct solver since a factorization of a shifted A may be too expensive. So one generally 
resorts to iterative methods to solve inner linear systems, called inner iterations. We call 
updates of approximate eigenpairs outer iterations. A combination of inner and outer 
iterations yields an inner-outer iterative eigensolver, also called an inexact eigensolver. 

The inexact inverse iteration and the inexact RQI are the simplest and most basic 
inexact eigensolvers. They not only have their own rights, but also are key ingredients of 
some other more sophisticated and practical inexact solvers, such as subspace iteration [15] 
and the Jacobi-Davidson method. So one must first analyze their convergence. This is the 
first step towards better understanding and analyzing those more practical inexact solvers. 

For A Hermitian or non-Hermitian, general theory on the inexact RQI can be found 
in Berns-Muller and Spence pQ, Simoncini and Elden |16j . Smit |18| . van den Eshof [21] 
and Xue and Elman [23J. Let ||r/%|| be the residual norm of the approximate eigenpair at 
outer iteration k. They prove that the inexact RQI converges cubically if £& = 0(||rjfc||) 
and quadratically if < £ < 1 with a constant £ not near one. For the inexact RQI, 
to the author's best knowledge, there is no result available on linear convergence and its 
conditions. However, the condition £& < £ < 1, though seemingly natural, is generally 
stringent. The author in [8] has revisited the convergence of the inexact RQI independent 
of iterative solvers and presents a new general convergence theory. It has been proved that 
the inexact RQI converges quadratically under a so-called uniform positiveness condition 
which is fundamentally different from and weaker than the condition £& < £ < 1. Based on 
it, the author has established a number of new results on the inexact RQI with MINRES. 
It appears that the method generally converges cubically for £/% < £ < 1 once £ is not 
near one, converges quadratically provided that £& = 1 — 0(||^fc||) and converges linearly 
provided that £"fc = 1 — 0(||rfc|| 2 ). In comparisons with the existing ones, these conditions 
are much more relaxed and can be exploited to implement the method more effectively than 
ever before. 

RQI and the inexact RQI are closely related to the simplified exact JD method with- 
out subspace acceleration and its inexact version for Hermitian and non-Hermitian cases, 
respectively; see [HJ ESJ H7J HHJ [21]. The two exact versions are mathematically equiva- 
lent [T7] , and the two inexact versions are equivalent too if an m + 1-step and m-step 
Galerkin-Krylov type method without preconditioner, e.g., the conjugate gradient method 
and the Lanczos (Arnoldi) method, is used for solving the linear systems arising in the 
inexact RQI and the inexact JD method at each outer iteration [16] , respectively. The sim- 
plified inexact JD method with a preconditioned Galerkin-Krylov solver is equivalent to the 
inexact RQI when the preconditioner is modified by a rank-1 matrix [4]. This equivalence 
is generalized to the exact and inexact two sided RQI and the simplified exact and inexact 
two-sided JD method [7] if the biconjugate gradient (BiCG) method without preconditioner 
is used to solve inner linear systems. 

In this paper, we revisit the inexact RQI with the (unpreconditioned and tuned pre- 
conditioned) Lanczos method for solving the inner linear systems. Although several results 
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have been established for the inexact RQI with Lanczos in the literature, one treats the 
residuals obtained by the Lanczos method as general ones and simply takes their norms 
but ignores their directions. A direct consequence is that all existing convergence results 
require < £ < 1, and fundamental effects of residual directions on convergence may have 
not been reasonably exposed. Remarkably, we first establish some attractive properties of 
the residuals obtained by the Lanczos method for the linear systems. By fully exploiting 
them, we take a different analysis approach to consider the convergence of the inexact RQI 
with Lanczos. Based on the new general convergence theory, we derive a number of insight- 
ful results that are not only stronger than but also fundamentally different from the ones 
available in the literature. 

For the inexact RQI with the unpreconditioned Lanczos, the most remarkable results 
we will prove are that the inexact RQI with Lanczos converges quadratically provided that 
(,k < (, with a constant £ that is allowed to be bigger than one and it converges linearly 
even if 3> 1 and is up to as large as 0(rJ L -rr). Therefore, in order to achieve quadratic 
convergence, we allow the Lanczos method to solve the linear systems with no accuracy 
in the sense of solving the linear systems. These results have a strong impact on effec- 
tive implementations of the method. For quadratic convergence, compared with prevailing 
implementations of the method, they indicate that the method now converges much more 
easily and the computational cost of solving the linear systems is generally reduced much 
more considerably. Numerical experiments demonstrate that our new implementation is 
twice to four times as fast as the prevailing implementations and the method still converges 
smoothly and quickly when increases up to 10 4 ~ 10 7 . 

As byproducts, similar to those done in 0[16], we establish lower bounds on the norms of 
approximate solutions wt+i of the linear systems obtained by the unpeconditioned Lanczos. 
We show that is always O( jijp ) no matter the inexact RQI with Lanczos converges 

cubically or quadratically. Therefore, it is distinctive that ||io^+x|| itself obtained by Lanczos 
cannot reveal the convergence behavior of the inexact RQI and cannot be used to design 
stopping criteria for inner iterations. Making use of these bounds, we present a simpler 
but weaker quadratic convergence result. Finally, as a global result, similar to that for the 
inexact RQI with MINRES [16], we derive a relationship between ||r&|| and ||rfc +1 ||, starting 
with an arbitrary vector instead of a reasonably good one. We will see that, unlike the 
exact RQI, the inexact RQI with Lanczos loses the residual monotonic decreasing property 
for an arbitrary starting vector. 

It appears [3j [23] that it is beneficial to precondition each shifted inner linear system 
with a tuned preconditioner, which can be much more effective than a usual one. We study 
the convergence of the inexact RQI with a tuned preconditioned Lanczos and show that 
our main theory in the unpreconditioned case can be nontrivially extended to the tuned 
preconditioned case. 

The paper is organized as follows. In Section [2j we review the inexact RQI and the 
new general convergence theory of [8] on the inexact RQI. In Section [3l we present conver- 
gence results on the inexact RQI with the unpreconditioned Lanczos Lanczos for solving 
inner linear systems. In Section HJ we extend the theory to the inexact RQI with a tuned 
preconditioned Lanczos method for solving inner linear systems. We perform numerical 
experiments to confirm our results in Section [5[ Finally, we end up with some concluding 
remarks in Section [6j 

Throughout the paper, denote by the superscript * the conjugate transpose of a matrix 
or vector, by I the identity matrix of order n, by || • || the vector 2-norm and the matrix 
spectral norm, and by A m i n , A max the smallest and largest eigenvalues of A, respectively. 
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2 The inexact RQI and general convergence theory 



RQI is a famous iterative algorithm and its locally cubic convergence for Hermitian problems 
is very attractive |13j . It plays a crucial role in some practical effective algorithms, e.g., the 
QR algorithm, [HI [13] . Assume that the unit length u k is a reasonably good approximation 
to x. Then the Rayleigh quotient 9 k = u* k Au k is a good approximation to A too. RQI [5113] 
computes a new approximation u k+ \ to x by solving the shifted inner linear system 

(A - 9 k I)w = u k (4) 

for Wk+i and updating u^+i = Wk+i/\\iVk+i\\ an d iterates until convergence. It is known 
P QUI Q3] that if 

I A — 6*n I < — min |A — A,-| 

2 j=2,3,..,n J 

then RQI (asymptotically) converges to A and x cubically. So we can assume that the 
eigenvalues of A are ordered as 

\\-e k \<\\ 2 -e k \<--- <\\ n -e k \. (5) 

With this ordering and A m i n < 9 k < A max , we have 

|A-0*| <~|A-A 2 |. (6) 

An obvious drawback of RQI is that at each iteration k we need the exact solution wt+i 
of (A — 9 k I)w = u k . For a large A, it is generally very expensive and even impractical to 
solve it by a direct solver due to excessive memory and/or computational cost. So we must 
resort to iterative solvers to get an approximate solution of it. This leads to the inexact 
RQI. @ is solved by an iterative solver and an approximate solution Wk+i satisfies 

(A - 9 k I)w k +i = u k + £ k d k , Uk+i = w k+ i/\\w k+ i || (7) 

with < £fc < £, where £ k d k with ||dfc|| = 1 is the residual of (A — 9 k I)w = u k , d k is the 
residual direction vector and £ k is the relative residual norm (inner tolerance) as ||ufc|| = 1 
and may change at every outer iteration k. This process is summarized as Algorithm 1. If 
£ k = for all k, Algorithm 1 becomes the exact RQI. 

Algorithm 1 The inexact RQI 
1: Choose a unit length uq, an approximation to x. 
2: for k = 0,1, ... do 
3: 9 k = u* k Au k . 

4: Solve (A — 9 k I)w = u k for w k+ \ by an iterative solver with 

|| (A - 9 k I)w k+1 - u k \\ = £ fc < £. 

5: u k+ i = w k+1 /\\w k+1 \\. 

6: If convergence occurs, stop. 

7: end for 



There are a number of general quadratic convergence results in the literature, e.g., 
[U [T6l [T8[ [21] , all under the same condition on < £ < 1 that is seemingly a very natural 
requirement. In [SJ, new general convergence results have been given under a new condition 
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that is fundamentally different and can relax very much. To present the results, we 
decompose u k and d k into the orthogonal direct sums 



u k = x cos 4> k + e k s\n4> k , e k ±x, (8) 
d k = x cos ip k + f k sin t/j k , f k ±x (9) 

with |jefc|| = \\f k \\ = 1 and 4>k = ^(u k ,x), ip k = Z(d k ,x). Here without loss of generality 
and for brevity, we suppose that (fi k is the acute angle between u k and x. Given this, we 
should stress that cos ip k is either positive or negative depending on d k . Note that ([7]) can 
be written as 

(A - 6 k I)w k+ i = (cos<^. + cos^ k )x + (e k sin^ fc + £ k fk sin^fe)- (10) 

Inverting A — 9 k I gives 

w k+ i = (A-6» fc )~ 1 (cos(/) fc + ^ cosip k )x + (A-e k I)~ 1 (e k sincj) k + £ k f k sin^fc). (11) 

Define ||rfc|| = ||(^4 — 6 k I)u k \\. Then by ([6]) we get | A2 — k \ > l* 2 ~^l . So it is known from 
[131 Theorem 11.7.1] that 

Ikfrll , 2||rzJ| . . 

< sin0 fc < " (12) 



Amax A m ; n |A 2 A| 

We comment that A max — A m i n is the spectrum spread of A and | A2 — A| is the gap or 
separation of A and the other eigenvalues of A. 
Before proceeding, we define 

n A max A mm 

1 |A 2 -A| 

throughout the paper. 

Theorem 1. |] 1/ the uniform positiveness condition 

I cos0 fc + ^cosV'fcl > c (13) 
is satisfied with a constant c > uniformly independent of k, then 

tan0 fc+ i < 20- — — -—sm z (p k (14) 

I cos <p k + £ fc cos ip k I 

< ^sin 2 </> fe + O(sin 3 fc ), (15) 
c 

that is, the inexact RQI converges quadratically at least for uniformly bounded £ k < £ with 
£ a moderate constant. 

Theorem 2. |8j If the uniform positiveness condition (I13p holds, then 

ll^ + ill<^\ll^|| 2 + 0(||r fc || 3 ). (16) 
c|A 2 - A| 

We make some comments on the above two theorems. 

Remark 1. They illustrate that it is the size of | cos 4> k +£, k cos ip k \ other than £ k < £ < 1 
that is critical in convergence. 

Remark 2. If £ k = for all k, then the inexact RQI reduces to the exact RQI and 
Theorems HH2] show the cubic convergence: taii(f) k+ i < ^§^7 sin 3 <f> k and = 0(||rfe|| 3 ). 



If the linear systems are solved with decreasing tolerance £k = 0(sm(frk) = 0(||rfc||), then 
we have the cubic convergence: tan^+i = 0(sin 3 (f>k) and ||?"fc+i|| = 0(||rfc|| 3 ). 

Remark 3. If cosipk is positive, the uniform positiveness condition holds for any 
uniformly bounded < £ with £ a moderate constant. So we may have £ > 1 considerably. 
If cos ipk is negative, the uniform positiveness condition | cos 4>k + £fc cos V'fcl ^ c means that 

C-COS(f> k 
COS V>fc 

if cos (f>h + cos ipk > c with c < 1 is required and 

c + cos^ fc 

-COS'i/'fc 

if — cos^fc — Ckcostpk > c is required. So the size of critically depends on that of 
cosV'fc, and for a given c we may have ~ 1 and even £^ > 1. Obviously, without 
the information on cos^fc, it would be impossible to access or estimate ^. As a general 
convergence result, however, its significance and importance consist in that it reveals a new 
remarkable fact: It appears first time that (13) may allow to be relaxed (much) more 
than that used in all known literatures and meanwhile preserves the same convergence rate 
of outer iteration. As a result, the condition < £ < 1 with constant £ not near one may be 
stringent and unnecessary for the quadratic convergence of the inexact RQI, independent of 
iterative solvers for the linear systems. The new condition has a strong impact on practical 
implementations as we must use a certain iterative solver, e.g., the very popular MINRES 
method and the Lanczos method (SYMMLQ) for solving (A — 8kI)w = u&. We will see that 
cosV'fc is critically iterative solver dependent. For Lanczos, cosifjk has some very attractive 
properties, by which we can precisely determine bounds for in Section El which are much 
more relaxed than those in the literature. For MINRES, we refer to [8j for cosipk and 
where cosV'fc and are fundamentally different from those obtained by Lanczos. 



3 Convergence of the inexact RQI with the unpreconditioned 
Lanczos 

The previous results and discussions are for general purpose, independent of iterative solvers 
for (A — 9kl)w = Ufc. Since we have A m ; n < 0^ < A max , the matrix A — 6^1 is Hermitian 
indefinite. The Lanczos method is a popular Krylov subspace iterative solver for Hermitian 
linear systems |14| . The method fits into the inexact RQI nicely, leading to the inexact RQI 
with Lanczos. 

We briefly review the Lanczos method for solving @. At outer iteration k, taking the 
starting vector v\ to be u k , the m-step Lanczos process on A — 6^1 can be written as 

(17) 

where the columns of V m = (v\, . . . ,v m ) form an orthonormal basis of the Krylov subspace 
K m (A - 9 k I,u k ) = lC m (A,u k ) and T m = (%) = V£(A - 6 k I)V m is an m x m Hermitian 
tridiagonal matrix |X3|, [T3] . 

Taking the zero vector as an initial guess to the solution of {A — 9kI)w = u k , the Lanczos 
method [6H121III] extracts the approximate solution w^+i = V m y to (A — 9kl)w = from 
K. m (A,Uk), where y is the solution of the Hermitian tridiagonal linear system T m y = e\ 
with ei being the first coordinate vector of dimension m. 

The Lanczos method is mathematically equivalent to the conjugate gradient method 
and has the optimality that the error of the approximate solution is minimal with respect 
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to the energy norm over the given Krylov subspace when the linear system is Hermitian 
positive definite [14] . For indefinite linear systems, it does not have any kind of optimality. 
For our case, the linear system (A — 0kl)w = u k is not only indefinite but also increasingly 
ill conditioned as Of. — > A. The system is typically ill conditioned and T m can be singular 
or nearly so, so that the Lanczos method may converge slowly and irregularly and can 
be typically big for m small. For more details, we refer to \12\ 114]. 

It is worth noting that we should naturally take m > 1; otherwise we would have 
)Ci(A,Uk) = span{nfc} and T\ = 0, so that the Lanczos method would break down and 
Wk+i would not exist. 

We will present convergence results on the inexact RQI with Lanczos. First of all, we 
present the following results. 



Theorem 3. It holds that 
and asymptotically 



cos ipk | < tan (frk , (18) 



sin^ fc > 1 - - sin 2 (f> k (19) 



by ignoring the higher order term 0(sin 4 (f> k ). 

Proof. Recall that the Lanczos method requires that its residual £, k d k satisfies £, k d k -L 
)C m (A, u k ). So, we specially have d k _L u^. Therefore, from (JSJ) and ([9]) we get 

cos (f> k cos ip k + elfk sin <j) k sin ip k = 0, 

which means 

|cOS^fc| # 

\e k fktaii(f) k \ < tan0 fc . 



sinV'fc 

From the above it follows that (|18p holds. By the Taylor expansion, from (|19p we get 
asymptotically 

sinV'A: = \A ~ c °s 2 i'k = l ~\\ cos2 + ^(cos 4 ipk) > 1 - 2 tan2 ^ fc = 1 ~ \ sin2 

by dropping the higher order term O(sin 4 0fc). □ 

Combining this theorem with (|14p of Theorem [H we can establish one of our main 
results for the inexact RQI with the preconditioned Lanczos. 

Theorem 4. If £^ < £ is uniformly bounded and satisfies 

(,sm<t> k <a <1 (20) 

wii/i a a constant, then the uniform positiveness condition ()13|) ZioWs and £/ie inexact RQI 
with Lanczos asymptotically converges quadratically at least: 

tan0 fc+1 < 2 ^ sin 2 fc . (21) 
1 — a 

converges cubically if 

£ fe = O(sin0 fe ); (22) 
converges at linear factor 7 ai /eas£: 

tan fc+1 < 7 sin 4> k (23) 



7 — 2/3 sin 2 



& < t^t^ ■ 1 < T^r-^—r ( 24 ) 



(2/3 + 7 )sin0 fc (2/3 + 7)sin0 fc 
toit/i 7 < 1 independent of k. 
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Proof. By (JT2J), we get 

sin <p k + i k sin tp k < sin (f> k + £k< sin <p k + £ • 
On the other hand, we get asymptotically from ([18p 



\ COS 4> k +ik COS > I COS (pk — ik \ cos Ipk 1 1 

> |1 - ^sin 2 ^ +O(sin 4 fe ) -^tan^fcl 
= |l-£fcsin0 fc | 

> 1 - £ sin <£ fc 

> 1 - a, (25) 

by dropping the higher order term O (sin 2 cpk), which is uniformly away from zero by as- 
sumption. So the uniform positiveness condition (|13p holds. We then derive asymptotically 
from (Q3D that 

, , , nn smcf)k+^ksm^k . 2 , 

tan0 fc+ i < 2/3- — — — r sm cp k 

| cos (p k + 6c cos Tpk | 

, na (sin^) fc +g fc ) . 2 

^ 2 Ph ? — ~ T7 sm W ( 26 ) 

, „(sin^> fc + . 2 , 

< 2/3 sm z (p k 

1 — a 

< 2/3-^— sin 2 </» A; + 0(sin 3 ^) 

1 — a 

which is just (|2T1) by ignoring O (sin 3 

Cubic convergence is direct from ([26]) if = O(sin0fc). 

It follows from (126p that the inexact RQI with Lanczos converges at linear factor 7 at 
least if for all k it holds that sin ^ < 1 and 

„ (sin <Ai. + fu) sin <Ai. 
1 - & sm 0& 



from which we get condition (124p by manipulation. □ 

Theorem [4] presents the conditions on cubic, quadratic and linear convergence in terms of 
a priori uncomputable sin (pp. . We next give alternatives of them in terms of the computable 
\\rk\\, so that they are of practical value as much as possible and can be used to control 
inner tolerance to achieve a desired convergence rate. 

Theorem 5. If ^ < £ is uniformly bounded and satisfies 

2£||r fc 



IA2-AI 



< a < 1 (27) 



with a a constant, the uniform positiveness condition holds and the inexact RQI with Lanc- 
zos asymptotically converges quadratically at least: 

l|r * +1 " - |A 2 -A|(|A 2 -A|-2 & |M) hl ' ■ (28) 

< &i±|^)| |rif . (29) 
(A 2 - A)^(l - a) 



It converges cubically if 

6 = 0(|H|); (30) 

it converges linearly: 

lkfc + i|| < Tlkfell (31) 
-T(A 2 - A) 2 - 16/9 2 ||7- fc |l 2 7 |A 2 -A| 

2|A 2 -A|(4/32+ 7 )||r fe || ^ (8/? 2 + 2 7 ) ||r fc || lJ j 

with 7 < 1 independent of k. 

Proof. Making use of (|12p gives 



< sin0 fc+ i < tan^fe+i, 



2||r fe || 2fllri.ll 
1 - £ fc sin <j) k > 1 - 4 " fc| > 1 - Ml *" > 1 - a > 

|A 2 — A| |A 2 — A| 

and 

sin (f) k + f fc < 2 ^ fc ^ + f fc . 

|A 2 — A| 

Substituting the above relations into ([26]) and (f2Tj) establishes ([28]) and ([29]) . respectively. It 



is clear from (|28p that the inexact RQI with Lanczos converges cubically once = 0(||rfc||). 
In order to make ||r&|| linearly converge to zero monotonically, from (|28|) we simply set 



8(A w -A) 2 2||r fc ||+g fc |A 2 -A| 
|A 2 -A|3 |A 2 -A|-2&[|r fc || 

with 7 independent of k. Solving it for fj. gives 



rjfcH < 7 < 1 



t 7 (A 2 -A) 2 -16/3 2 ||r fc || 2 7 |A 2 - A 



2|A 2 -A|(4/? 2 + 7 )|N| (8/3 2 + 2 7 )||r fc ||- 

□ 

We make some comments on Theorems EHHJ 

Remark 1 . The quadratic convergence condition (|20|) in Theorem H] is sufficient and 
it indicates that £ > 1 considerably is allowed. Noting that (|20p is assumed to satisfy for 



all k, it is sufficient that £ < gir] * , . So the old requirement £fc < £ < 1 is stringent and not 
necessary for quadratic convergence. The comments apply to (|27p in Theorem [5] as well. 
Note that these two conditions are only sufficient but not necessary. Actually, it can be 
seen from (|26p that £ < sin C o with a moderate constant c, say, 1 < c < 5 also works well 
for quadratic convergence. The numerical experiments in Section [5] have confirmed this. 
Since the initial residual norm is = 1 before performing the Lanczos method, in order 
to make the inexact RQI with Lanczos converge quadratically, we do not need to solve the 
inner linear systems (A — 6k)w = u^s with any accuracy at all in the sense of solving the 
linear systems. 

Remark 2. Conditions (|24p and (|32p for linear convergence show that £& can be as big 
as 0( ) and Of-n— it) as outer iterations proceed. More precisely, (|24p indicates that the 

sin || Tfc || 

inexact RQI with Lanczos still converges linearly at least even if £& is as big as 0( sin< , ) 
with the order constant OQ 7 , smaller than one. 
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Below, as done in [16] , we estimate in ([7]) obtained by Lanczos and present 

more results. Note that the exact solution of (A — 9 k I)w = u k is Wk+i = (A — Okl)^ 1 ^, 
which corresponds to = in (jlip . Therefore, we have from (jlip and (|12p 

cos ( ' 



\w k +i\ 



i 



\9 k 

o 



M 
i 



sin 2 4> k 



+ 0(sin (j> k ) 

= \\(A-e k i)- 1 
-- o 



i 



From (fTTj) and (fT2|) . we also see that these estimates hold for £ k = O(sin0fc) = 0(||rfc||). So 
||u;fc+i|| is also 0(|p-i|) 2 ) when the inexact RQI with Lanczos converges cubically. 

II 'fa II 

Theorem 6. Assume that £ k < £ is uniformly bounded and satisfies £sm(fi k < a < 1 with 
a a constant. Then asymptotically we have 

;i-a)|A 2 -A| 



\Wk+l\ 



Ffc+il 



> 



< 



< 



4/3||r fc | 

n^fc+iii 



Wj + g 2 

|A 2 -A|(l-a) 



(33) 
(34) 
(35) 



where (I34D ZioWs strictly. 

Proof. As ^dfc _L Ufc and (A - 6 k I)w k+ i = u k + we have 

||(^-^JK + i|| 2 = ||u fe || 2 + ||a4|| 2 

So we get from = ^fc+i/ll^fc+i II that 



1+& 



||(A-0 fc 7>fc+i|| 



\Wk+l\ 



By the optimality of Rayleigh quotient we obtain 



ll r fc+i 
which shows 



|| (A - fc+ ii> fe+ i|| <\\(A- 9 k I)u k+1 



\Wk+l\ 



\Wk+l\ 



By using ([TT 



It is easy to verify (cf. [131 P- 77]) that 

|A 2 - A | sin 2 (j) k < | A - k \ < (A max - A min ) sin 2 4> k . 
(J35J), d3SJ) and ([T2j) in turn, we obtain 

| COS (f) k + £fc cos ljj k \ 



(36) 

(37) 
(38) 



\Wk+l\ 



> 
> 
> 
> 
> 
> 



\e k -x\ 

[1 - £ fc sin^fcl 

\e k - A| 

1 — a 



\e k - a| 



i 



a 



(Amax A m j n ) sin 

(1 -q)|A 2 - A| 



l r fc|| 

l-a)|A 2 -A| 



m\r k \ 



1 n 



which proves (f33|) . Substituting (f33|) into ([34")) establishes ([35]) . 



□ 



(j35[) indicates that the inexact RQI with Lanczos converges quadratically if a is not near 
one. By combining this theorem with Theorems (3HHJ (|33p shows that is always no 

less than 0( H Km ) when the inexact RQI with Lanczos converges quadratically provided that 

Ir fell 

a is not near one. Noting that the inexact RQI with Lanczos for = 0(sm(j) k ) = 0(||rfc||) 
converges cubically and is also 0( M ,, a ) (cf. the comments before Theorem [6]), 

this illustrates that the size of ||iOfe+i|| itself cannot reveal cubic and quadratic convergence 
rates of the inexact RQI with Lanczos. However, we cannot recover the cubic convergence 
of the exact RQI and the inexact RQI when = £ = and = £ = 0(sm(p k ) = 0(||rfc||), 
respectively. So (j35|) is weaker than Theorems HHSJ This is because < \\(A — 

8kl)uk+i\\ is not sharp in the proof. 

So far, all the convergence results are local, that is, they care how the exact and inexact 
RQI behaves only from the current outer iteration to the next one, assuming that current 
(9k, Uk) is already a reasonably good approximation to (A, x). As is well known, one of the 
important properties of the exact RQI is its global residual monotonic decreasing property, 
i.e., Ilnt+ill < \\rk\\, for any (poor) starting vector uq; see Theorem 4.8.1 of [131 P- 79]. We 
now present a global property to the inexact RQI with Lanczos. 

Theorem 7. For the inexact RQI with Lanczos starting with any starting vector uq, we 
have 



Wk+iW < \Jl + e k \Vkl A: = 0,1,.... (39) 

Proof. From ([7]), we have 

w k+ i = (A- 6 k I)~ l {uk + 6A). 

Again, note that for the Lanczos method its residual £fc(4 satisfies ^kd^Uk = 0. Then from 
(|37p and the Cauchy-Schwarz inequality, we get 



kfc+ij < \\{A-e k i)u k+1 \\ = y^+jfc 



2 



Fit II Until ||u7 fc +i|||Ffcl 



|| (A - e k I)-^{u k + tkd k )\\\\{A - e k i)u k \ 



< 



\{u k +i k d k )*{A - 9 k I)- l {A - 9 k I)u k \ 



1 + 3 



i+e 



|1 + ikd* k u k \ 

which proves (f39l) . □ 

This theorem shows that, unlike RQI, ||rj.|| obtained by the inexact RQI with Lanczos 
is not monotonic decreasing in the global sense for an arbitrary starting vector uq. We have 
made numerical experiments on some matrices for some starting «o' s generated randomly 
and found that it is indeed case. 



1 1 



4 The inexact RQI with a tuned preconditioned Lanczos 



We have found that for a given £ satisfying our convergence conditions, we may still need 
quite many inner iteration steps at each outer iteration. This is especially the case for 
difficult problems, i.e., big /3's, or for computing an interior eigenvalue A since it leads 
to a highly Hermitian indefinite matrix (A — 9 k I) at each outer iteration. So, in order 
to improve the overall performance, preconditioning may be necessary to speed up the 
Lanczos method. Some preconditioning techniques have been proposed in e.g., [HHE]. In 
the unpreconditioned case, the right-hand side u k of is rich in the direction of the desired 
x. We can benefit much from this property when solving the linear system. Actually, if the 
right-hand side is an eigenvector of the coefficient matrix, Krylov subspace type methods will 
find the exact solution in one step. However, a usual preconditioner loses this important 
property, so that inner iteration steps may not be reduced [U H |]. A preconditioner 
with tuning is necessary to recover this property and meanwhile attempts to improve the 
conditioning of the preconditioned system, so that considerable improvement over a usual 
preconditioner is possible [31 El [23]. In what follows we show how to extend our previous 
theory to the inexact RQI with a tuned preconditioned Lanczos. 

Let Q = LL* be a Cholesky factorization of some Hermitian positive definite matrix 
which is an approximation to A — 9 k I in some sense [U El [23]. A tuned preconditioner 
Q = CC* can be constructed by adding a rank-1 or rank-2 modification to Q, so that 

Qu k = Au k ; (40) 

see [3l [23] for details. Using the tuned preconditioner Q, the shifted inner linear system 
([4]) is equivalently transformed to the preconditioned one 

Bw = C' l (A-6 k I)C~*w = (41) 

with the original w = C~*w. Once the Lanczos method is used to solve it, we are led to the 
inexact RQI with a tuned preconditioned Lanczos. A power of the tuned preconditioner Q 
is that the right-hand side C~ l Uk is rich in the eigenvector of B associated with its smallest 
eigenvalue and has the same quality as Uk as an approximation the eigenvector x of A, while 
for the usual preconditioner Q the right-hand side L~ l iik does not possess this property. 

Take the zero vector as an initial guess to the solution of (j4"TT) and let w k+ \ be the 
approximate solution obtained by the m-step Lanczos method applied to it. Then we have 

C-\A - 9 k I)C-*w k+l = £-W + 44, (42) 

where w k+ \ € IC m (B, C~ 1 u k ), 6, k dk with ||c4|| = 1 is the residual and d k is the residual 
direction vector. Keep in mind that w k+ \ = C~*w k+ \. We then get 

(A - 6 k I)w k+ i = u k + i k Cd k = u k + £k\\£d k \\ ■ (43) 

So £fc and d k in ([7D are £fc||£djj| and / , respectively. Hence our general Theorems [TH2] 

\\C-o.k\\ 

apply and is not repeated here. 

How to extend Theorem [3] to the preconditioned case is nontrivial and need more work. 
Let (ni,yi), i = 1, 2, . . . , n be the eigenpairs of B with 

I fJ-l \<\ 1*2 \< ■ ■ ■ <| Mn I ■ 

Define u k = £ _1 n/ c /||£ _1 Ufc||- Similar to ([HJ) and Q, let 

u k = 2/icos0 fc + e fe sin^, e k J_ yi, \\e k \\ = 1, (44) 
dk = Vicosipk + / fc sinV; fc , f k _L y u \\f k \\ = 1 (45) 

1 o 



be the orthogonal direct sum decompositions. Then it is known [5] that 

|Mi| = O(sinfo), (46) 
sin <p k < c\ sin 4> k (47) 

with ci a constant. 

From Theorem [3] and (147h . the following results are direct. 

Lemma 1. It holds that 

I cos ijj k | < tan cp k , (48) 

and asymptotically 

sin^ fc > 1 - r sin2 <^ ( 49 ) 
fry ignoring the higher order term 0{sm A <p k ). 

Remark. Combining with ()47p . this theorem means 

| cos^fcj = O(sin^fe), sin^fc = 1 — 0(sin 2 ^). (50) 
Theorem 8. It holds that 

|cosVVc| = 0(sin<£ fc ), (51) 
sin^fc = l-0(sin 2 ^). (52) 



Proof. From the requirement of the m-step Lanczos for (142ft . we have £kdk -L fcm(B, £ 1 u k ). 
Particularly, it holds that 



<i££ 1 u k = dlC 1 u k = 0, 



Cd, 



k_ 



from which and u k = £ u k \\ it follows that djJlMfc = 0. Therefore, from d k . , . 

we have 

= d* k u k = d* k C*£T^u k = WCdkWWC^UkWdlC'*^ = 0, 
I 

i.e., 

d* k £~*u k = 0. (53) 

By definition, we have 

£-\A-0 k I)£-*y 1 = /il y 1 , 

from which it follows that 

(A - 9 k I)u k = Ati 1 

*yx\\ 

with -Ufc = £ _ *yi/||£ _ *yi||. Therefore, by standard perturbation theory and (06]), we get 

sinZ(n fc ,x) = 0(Ml JI « ,. ) = O(lwl) = O(sin^). (54) 
l|£ 2/1 II 

On the other hand, from (|47p . we have 

sin^ fc = sinZ(-u fc ,yi) = sin Z(£ _1 u fc , yi) = 0(sin<£fc). 
Therefore, we can write 

"A: = 2/1 + O(sin0 fc ), 



which leads to 



C *u k = C *yi + O(sin0jfe). 



Thus, we have 



sinZ(£ *u fc ,£ *yi) = sinZ(£ *u fc , u k ) = 0(sin fc ). 



(55) 



Since 



Z(£ *-u fc ,x) < Z(£ *u k ,u k ) + Z(u k ,x) 



combining ([54"|) and ([55]) . we get 



sinZ(£ x) < sin Z(£ *u&, + sin Z(%, x) = 0(sin </>fc). 



(56) 



Recall that d k = x cos V'fc + sin ip k and substituting it and the orthogonal direct sum 
decomposition 



with g k _L x into (153|) . Then based on (153|) and following the proof of Theorem [3l we get 



Using this theorem and writing (I51j) as | cos V>£:| < C2 sin</>fc with C2 a constant, it is direct 
to extend Theorems HHS in the unpreconditioned Lanczos case to the tuned preconditioned 
Lanczos case. We have made preliminary numerical experiments and confirmed the the- 
ory. Since our concerns in this paper are the convergence theory of the inexact RQI with 
the unpreconditioned and tuned preconditioned Lanczos and the pursue of effective tuned 
preconditioners are beyond the scope of the current paper, we will only report numerical 
results on the inexact RQI with the unpreconditioned Lanczos. 

5 Numerical experiments 

Our numerical experiments were performed on an Intel (R) Core (TM)2 Quad CPU Q9400 
2.66GHz with main memory 2 GB using Matlab 7.8.0 with the machine precision e = 
2.22 x 1CP 16 under the Microsoft Windows XP operating system. 

We report numerical experiments made by the inexact RQI with the unpreconditioned 
Lanczos on four symmetric (Hermitian) matrices: BCSPWR08 of order 1624, CAN1054 of 
order 1054, DWT2680 of order 3025 and LSHP3466 of order 3466 [2j. Note that the bigger 
the factor f3 is, the worse conditioned x is. At the meanwhile, for (3 big, Theorem [1] and 
Theorem H] show that although RQI and the inexact RQI can still converge cubically and 
quadratically, they may converge more slowly and needs more outer iterations as the factors 
2(3 and ^| in {HD and (HQ are big. As a reference, we use the Matlab function eig.m to 
compute (3. We find that DWT2680 and LSHP3466 are considerably more difficult than 
the two others. We only report the results on the computation of the smallest eigenpair. 

Remembering that the cubic convergence of the inexact RQI with Lanczos for = 
O(ll^fcll) occurs when updating (9k,u k ) to get (0k+i,u k +i), in the experiments we take 



C *Uk = \\C *-Ufc||(xcosZ(£ *u k , x) + g k sin Z(£ *u k ,x)) 



cosi/>fc| < |tanZ(£ *u k ,x)\, 
simp k = 1 — 0(sin 2 Z(£~*u k , x)). 



Combining them with (j56|) yields (|5ip and (|52p . 



□ 



We construct the same initial uq for each matrix that is x plus a reasonably small 
perturbation generated randomly in a uniform distribution, such that \X — 8q\ < ^ 2 ~^\ . 
The algorithm stops whenever ||rjfe|| = ||(^4 — 9kl)uk\\ < ||-A||iioZ, where tol = 10~ 14 unless 
stated otherwise. In the experiments, we use the Matlab function symmlq.m to solve the 
inner linear systems with < £ < 1. We should notice that for £j. > 1 the Matlab 
function symmlq.m cannot be applied. Since the Lanczos method behaves irregularly and 
may nearly break down or break down for indefinite linear systems, that is, T m in ([3]) is ill 
conditioned and can be nearly singular and even numerically singular, it may produce bad 
approximate solutions with large norms and large residual norms £fc's for some steps m. As 
far as solving the linear systems is concerned, such approximate solutions have no accuracy 
and no practical value. In symmlq.m, if such a bad approximate solution emerges, it always 
outputs the approximate solution as zero and the residual norm = 1 simply, telling us 
nothing! However, we have seen that in the inexact RQI with Lanczos, > 1 is allowed. 
So for our purpose, we have worked out a Lanczos code that uses the Gram-Schmidt with 
iterative refinement [20] to generate a numerically orthonormal basis of the Krylov subspace 
JC m (A, Uk) and delivers 'correct' results that the Lanczos method should produce. We point 
out that our Lanczos code is not optimized but numerically stable. 

We test the inexact RQI with Lanczos by choosing ^ as in (I57p and a few fixed £k = 
£ = 0.1,1,5,20. Based on our theory, the method should converge quadratically for the 
£ = 1,5,20 and use almost the same outer iterations as those for £ = 0.1. Therefore, the 
total computational cost may be reduced considerably. For the inexact RQI with Lanczos, 
the total inner iteration steps, i.e., the total matrix- vector products in inner iterations, is 
a good measure of overall performance of the method. 

Tables [THU list the computed results, where iters denotes the number of total inner 
iteration steps and iter'** -1 ) is the number of inner iteration steps when computing 
the "-" denotes the stagnation of symmlq.m at the iter^ k ~ l "> -th step, and res^ k ~^ is the 
actual relative residual norm of the inner linear system. Clearly, iters is a reasonable 
measure of the overall efficiency of the inexact RQI with Lanczos. We comment that in 
symmlq.m the output iter^^ 1 ^ = m — 1, where m is the steps of the Lanczos process. 

Before commenting the experiments, we should remind that in finite precision arith- 
metic ||rfc||/|| can not decline further whenever it reaches a moderate multiple of e = 
2.2 x 10~ 16 . Therefore, assuming that the algorithm stops at outer iteration k, if sin0£._i 
or ||rfc_i|| < 10~ 6 or 10~ 9 , then the algorithm may not continue converging cubically or 
quadratically at the final outer iteration k. Another point is that when judging convergence 
rates, we must take the factor (3 into account. The smaller it is, the more clearly cubic and 
quadratic convergence exhibits, as indicated by (|26|) and (|2"8]); the bigger it is, the less ap- 
parent cubic and quadratic convergence is. So we should precisely base (l26|) or (f28l) to judge 
cubic and quadratic convergence of sin^fc or ||rjfe||. In this sense, we see from Tables HHH 
that the exact RQI and the inexact RQI with Lanczos for decreasing £^ converge cubically 
and the method starts to converge quadratically for the given fixed £'s after very few outer 
iterations. But for £ > 1, the method is much more efficient than that for £ = 0.1 and for 
decreasing the method with £ > 1 is four times and twice as fast as that with £ = 0.1 
for BCSPWR08 and for CAN1054 and DWT2680, respectively. For LSHP3466, the gain is 
not so great, but the method with £ > 1 is still one and a half times as fast as the method 
with £ = 0.1. With the fixed £ = 1,5, 20, it is always three to ten times as fast as that with 
decreasing = 0(||rfc||) for the four test matrices. It is seen that for a bigger £ the method 
may need a little more outer iterations but it does indeed converge quadratically and is in 
agreement with quadratic convergence bound (I26h . Why the method with fixed bigger £fc's 
converges a little more slowly is due to the bigger convergence factor in (|21|). 

Note that the linear systems (A — 6k)w = u^'s are Hermitian indefinite and become 

1 K 



< £ 


k 


Ikfell 


sin fe 


res y > 


iter yK 1} 


iters 


(RQI) 


1 

2 
3 


0.0124 
3.1e -8 
2.0e - 15 


0.0036 
8.5e - 8 
6.6e - 15 








ll r fe-ill 


1 

2 


0.0071 
3.7e - 8 


0.0029 
1.2e - 7 


0.0367 
4.1e -4 
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40 


1003 




3 


3.0e - 15 


3.0e - 15 


- 
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0.1 
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0.0090 


0.0045 


0.0950 
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2.1e - 5 


3.7e - 5 


0.0847 


24 
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3.3e - 13 


3.2e - 13 


0.0754 
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0.0462 


0.0165 


0.7161 
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87 
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5.1e -4 


8.9e - 4 


0.8505 
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2.3e - 7 


2.9e - 7 


0.9829 
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Lie - 14 


1.3e - 14 
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0.1259 


0.0332 


2.0694 
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87 
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0.0107 


0.0064 


3.0112 
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4.2267 


13 






4 


1.2e - 7 
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4.6e - 6 
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1.6e - 10 


2.2e - 10 


13.9482 
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outer iterations 


iters 


5 


110 


550 


10 


21 


210 


15 


10 


150 


20 


7 


140 


30 


5 


150 



Table 1: BCSPWR08, f3 = 40.19, sin, 



0.1020. 



increasingly worse conditioned and even numerically singular as 6k — > A with increasing k. 
So, more inner iteration steps are needed generally for a fixed £ as k increases. We find 
that for the difficult DTW2680 and LSHP3466, many more iters's are used than those for 
BCSPWR08 and CAN1054. 

We have tested many fixed £'s ranging from 1 to 50 for each matrix and found that 
the method asymptotically converges quadratically. £ > 10 does not satisfy the sufficient 
condition (|20p for quadratic convergence but is a moderate multiple of ■ The algorithm 
with these £ behaves almost the same as that with £ = 1,5 and uses a little more outer 
iterations and comparable total inner iteration steps iters. 

For linear convergence, we see that conditions (|24p and (|32p depend on /?, which is 
unknown a priori. So it appears impossible to design a practical criterion robustly and 
reliably unless a rough estimate on /3 is available in advance. Note that the convergence 
of the method allows to increase up to o7icnn as outer iterations proceed. Therefore, 
we may not care £fc's themselves but implement the inexact RQI with Lanczos for certain 
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Table 2: CAN1054, (3 = 88.28, sine 
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Table 3: DWT2680, tol = 10~ 12 , p = 2295.6, sin</> = 0.1095. 
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0.0948 


256 






4 


5.3e - 13 


5.6e - 11 


0.0965 


287 




1 


1 


0.0408 


0.0251 


0.5335 


3 


424 




2 


6.4e - 4 


0.0036 


0.9685 


15 






3 


7.6e - 7 


1.9e - 5 


0.9770 


123 






4 


2.7e - 12 


2.9e - 11 


0.9907 


283 




5 


1 


0.0779 


0.0370 


1.0528 


2 


444 




2 


0.0088 


0.0128 


4.2424 


5 






3 


2.4e - 4 


0.0021 


4.4646 


21 






4 


7.0e - 7 


2.8e - 5 


4.6415 


121 






5 


1.9e - 11 


2.4e - 10 


4.6506 


146 






6 


l.Oe - 14 


3.8e - 13 


4.6805 


149 




20 


1 


0.0779 


0.0370 


1.0528 


2 


427 




2 


0.0245 


0.0195 


12.8944 


3 






3 


0.0041 


0.0089 


15.1663 


5 






4 


3.4e - 4 


0.0030 


19.9916 


16 






5 


4.7e - 6 


8.6e - 5 


19.3891 


92 






6 


1.9e -9 


5.6e - 8 


19.9778 


145 






7 


1.5e - 14 


4.7e - 13 


19.7118 


164 





m 


outer iterations 


iters 


10 


278 


2780 


20 


54 


1080 


30 


32 


960 


40 


19 


760 


50 


13 


650 


60 


11 


660 



Table 4: LSHP3466, tol = 10~ 12 , f3 = 2613.1, sin^ = 0.1011. 



1 n 



fixed small inner iteration steps m's, expecting that resulting ^'s are not too large and at 
least obey one of (i24|) and ([32]) . Of course, if T m is too ill conditioned, it is possible for 
to be too large. We have tested several m's for each test matrix; see Tables HHH Figure Q] 
displays convergence processes of the inexact RQI with Lanczos for various fixed m's. 

We find that the method works very well and robustly. In terms of iters, it is seen from 
the tables that the overall efficiency of the inexact RQI with Lanczos for fixed small m's is 
comparable to that of the method with given fixed £'s, except for m = 5 for BCSPWR08 
and CAN1054 and m = 10 for DWT2680 and LSHP3466. Furthermore, we observe that 
for BCSPWR08 and CAN1054 the inexact RQI with Lanczos converges almost as fast as 
the exact RQI for m = 30, and for the difficult problems DWT2680 and LSHP3466 we 
need to properly increase m to achieve fast outer convergence. As explained, it is not 
surprising from the figures that a five-step and at most ten-step Lanczos method for the 
inner linear systems is enough to ensure the convergence of the inexact RQI with Lanczos. 
However, for the difficult BWT2680 and LSHP3466, the inexact RQI with Lanczos fails 
to converge when m = 5. The reason is that some £fc's are too big and violate the linear 
convergence conditions. Finally, we should point out that although Lanczos with a smaller 
m usually produces a bigger and thus makes the inexact RQI use more outer iterations 
than Lanczos with a bigger m, the total inner iteration steps iters used by the former is 
not necessarily more than that by the latter. 



slr»|> k 





outer iterations used by inexact RQI with Lanczos for BCSPWR08 



20 40 60 80 100 120 

outer iterations used by inexact RQI with Lanczos for CAN1054 




50 100 150 200 250 300 

outer iterations used by inexact RQI with Lanczos for DWT2680 




50 100 150 200 250 300 

outer iterations used by inexact RQI with Lanczos for LSHP3466 



Figure 1: The inexact RQI with Lanczos for varying inner iteration steps 



To see how big £fc's may be as outer iterations converge, we display the curves of £/%'s 
versus sin 1 ( ^, fc 's for some fixed m's in Figure [2j We remark that ^ in the figure should 
correspond to £k-l m our context. 

It is clear from Figure [2] that the inexact RQI with Lanczos works very well and sin <j)k 



on 



q k versus 1/sinit> k 



4 k versus 1/sin^ 




outer iterations for BCSPWR08 
q k versus 1/Sln^. 





outer iterations for CAN1054 
!; versus 1/sin^ 




outer iterations for DWT2680 



outer iterations for LSHP3466 



Figure 2: & versus 



tends to zero smoothly and quickly, that is, sin( ^, fc tends to infinity smoothly and quickly, 
though most of ^'s are much bigger than one and some of them are near si ^ ( j >k and can be 
as big as 10 4 ~ 10 7 ! Furthermore, we observe that almost all ft-'s are smaller than . 1 , 

and the only exception is £q > S i^ 6 ^ or BWT2680. 

We also computed some other eigenpairs of each test matrix. We observed similar 
behavior and confirmed our theory. However, when an interior eigenpair was required, we 
often needed much more iters. This is because the shifted inner linear systems can be 
highly indefinite (i.e., each shifted matrix has many positive and negative eigenvalues) and 
may be hard to solve. If the smallest or largest eigenpair is required, then A — 6kl has 
only one negative or positive eigenvalue, assuming that A only has simple eigenvalues. As 
a consequence, after the smallest Ritz value converges to the smallest eigenvalue A — 9^ of 
A — 0}~I, the Lanczos method will behave as if A — 9^1 is positive or negative definite, so 
that it converges smoothly after the smallest Ritz value converges. Comparing with the 
computation of the smallest or largest eigenpair, we found that we must take a considerably 
bigger fixed inner iteration steps m to make the method converge correctly when an interior 
eigenpair was desired. For a descriptive analysis, see, e.g., [3| [5l [T2"l [23] . 



6 Conclusions 



We have considered the convergence of the inexact RQI with the unpreconditioned and 
tuned preconditioned Lanczos methods and have established a number of results. These 
results show how inner tolerance affects accuracy of outer iterations and provide practical 



criteria on how to best control inner tolerance to achieve quadratic convergence of the 
inexact RQI. It is the first time to appear surprisingly that the inexact RQI with Lanczos 
converge quadratically provided < £ with £ a constant being allowed bigger than one. 
This is both attractive and exciting as we can implement the method much more effectively 
than ever before. Numerical experiments have confirmed our theory. 

Perspectively, since the inexact RQI has intimate relations with the simplified Jacobi- 
Davidson method and the former is mathematically equivalent to the latter when a Galerkin- 
Krylov type solver, e.g., the Lanczos method, is used for solving the linear systems, we 
can use the convergence theory developed here for the inexact RQI to help understand 
the inexact simplified JD method. Meanwhile, the inexact inverse iteration is a simpler 
variation of the inexact RQI, where varying O^s are fixed to be a constant a, leading to 
different convergence behavior. Thus, a specific analysis is needed. It is likely to exploit 
the analysis approach used in this paper to study the inexact inverse iteration. Finally, 
although we have restricted to the Hermitian case, the analysis approach in this paper 
might be applied to the inexact RQI for the non-Hermitian eigenvalue problem, where the 
Arnoldi method is used for non-Hermitian inner linear systems. 

We have only considered the standard Hermitian eigenvalue problem Ax = Xx in this 
paper. For the Hermitian definite generalized eigenvalue problem Ax = XMx with A 
Hermitian and M Hermitian positive definite, if the M-inner product, the M-norm and the 
M -norm, the angle induced from the M-inner product are properly placed in positions 
of the usual Euclidean inner product, the Euclidean norm and the usual angle, then based 
on the underlying M-orthogonality of eigenvectors of the matrix pair (A, M) we are able to 
extend our theory developed in the paper to the inexact RQI with the unpreconditioned and 
tuned preconditioned Lanczos for the generalized problem. This work is under research. 

References 

[1] J. Berns-Miiller, I. G. Graham and A. Spence, Inexact inverse iteration for symmetric 
matrices, Linear Algebra Appl, 416 (2006), pp. 389-413. 

[2] I. S. Duff, R. G. Grimes and J. G. Lewis, User's Guide for the Harwell-Boebing sparse 
matrix collection (Release 1), Tech. Rep., RAL-92-086, Rutherford Appleton Labora- 
tory, UK, 1992. Data available at http://math.nist.gov/MarketMatrix, 

[3] M. A. Freitag and A. Spence, Convergence of inexact inverse iteration with application 
to preconditioned iterative solves. BIT, (47) (2007), pp. 27-44. 

[4] M. A. Freitag and A. Spence, Rayleigh quotient iteration and simplified Jacobi- 
Davidson method with preconditioned iterative solves, Linear Algebra Appl, 428 
(2008), pp. 2049-2060. 

[5] M. A. Freitag and A. Spence, A tuned preconditioner for inexact inverse iteration 
applied to Hermitian eigenvalue problems, IMA J. Numer. Anal, 28 (2008), pp. 522- 
551. 

[6] G. H. Golub and C. F. van Loan, Matrix Computations, The John Hopkins University 
Press, Baltimore, London, 1996. 

[7] M. E. Hochstenbach and G. L. G. Sleijpen, Two-sided and alternating Jacobi-Davidson, 
Linear Algebra appl, 358 (2003), pp. 145-172. 

[8] Z. Jia, On convergence of the inexact Rayleigh quotient iteration without and with 
MINRES, arXiv: math/0906.2238v3, submitted. 



[9] Y. Notay, Combination of Jacobi-Davidson and conjugate gradients for the partial 
symmetric eigenproblem, Numer. Linear Algebra AppL, 9 (2002), pp. 21-44. 

[10] Y. Notay, Convergence analysis of inexact Rayleigh quotient iteration, SIAM J. Matrix 
Anal. AppL, 24 (2003), pp. 627-644. 

[11] Y. Notay, Inner iterations in eigenvalue solvers, Report GANMN 05-01, Service de 
Metrologie Nucleaire, Universite Libre de Bruxelles, Belgium, 2005. 

[12] C. C. Paige, B. N. Parlett and H. A. van der Vorst, Approximate solutions and eigen- 
value bounds from Krylov subspaces, Numer. Linear Algebra AppL, 2 (1995), pp. 115- 
134. 

[13] B. N. Parlett, The Symmetric Eigenvalue Problem, SIAM, Philadelphia, PA, 1998. 

[14] Y. Saad, Iterative Methods of Large Sparse Linear Systems, 2nd Edition, SIAM, 
Philadelphia, PA, 2003. 

[15] M. Robbe, M. Sadkane and A. Spence, Inexact inverse subspace iteration with precon- 
ditioning applied to non-Hermitian eigenvalue problems, SIAM. J. Matrix Anal. AppL, 
31 (2009), pp. 92-113. 

[16] V. Simoncini and L. Elden, Inexact Rayleigh quotient-type methods for eigenvalue 
computations, BIT, 42 (2002), pp. 159-182. 

[17] G. L. G. Sleijpen and H. A. van der Vorst, A Jacobi-Davidson iteration method for 
linear eigenvalue problems, SIAM Rev., 42 (2000), pp. 267-293. 

[18] P. Smit and M. H. C. Paardekooper, The effects of inexact solvers in algorithms for 
symmetric eigenvalue problems, Linear Algebra AppL, 287 (1999), pp. 337-357. 

[19] A. Stathopoulos and Y. Saad, Restarting techniques for the (Jacobi-)Davidson eigen- 
value methods, Electr. Trans. Numer. Anal., 7 (1998), pp. 163-181. 

[20] G. W. Stewart, Matrix Algorithms Vol. II: Eigensystems, SIAM, Philadelphia, PA, 
2001. 

[21] J. van den Eshof, The convergence of Jacobi-Davidson iterations for Hermitian eigen- 
problems, Numer. Linear Algebra AppL, 9 (2002), pp. 163-179. 

[22] H. A. van der Vorst, Computational Methods for Large Eigenvalue Problems, In P. 
G. Ciarlet and J. L. Lions (eds), Handbook of Numerical Analysis, Vol. VIII, North- 
Holland, Elsevier, pp. 3-179, 2002. 

[23] F. Xue and H. Elman, Convergence analysis of iterative solvers in inexact Rayleigh 
quotient iteration, SIAM. J. Matrix Anal. AppL, 31 (2009), pp. 877-899. 



